Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Electron start-up performance script #10442

Merged

Conversation

cdamus
Copy link
Contributor

@cdamus cdamus commented Nov 19, 2021

What it does

Adds a new performance script for measurement of frontend start-up performance of the Electron example, similar to what was done in PR #9777. In fact, much of that script is factored out into a module of reusable functions to build the Electron script. This new script runs in much the same way, with similar command-line arguments to specify the number of runs and where to put the Chromium trace files.

Resolves #10440.

Example output:

[Performance][Electron Frontend Startup][1] Largest Contentful Paint (LCP): 6.716 seconds
[Performance][Electron Frontend Startup][2] Largest Contentful Paint (LCP): 6.037 seconds
[Performance][Electron Frontend Startup][3] Largest Contentful Paint (LCP): 5.992 seconds
[Performance][Electron Frontend Startup][4] Largest Contentful Paint (LCP): 6.026 seconds
[Performance][Electron Frontend Startup][5] Largest Contentful Paint (LCP): 6.052 seconds
[Performance][Electron Frontend Startup][6] Largest Contentful Paint (LCP): 6.137 seconds
[Performance][Electron Frontend Startup][7] Largest Contentful Paint (LCP): 6.081 seconds
[Performance][Electron Frontend Startup][8] Largest Contentful Paint (LCP): 5.964 seconds
[Performance][Electron Frontend Startup][9] Largest Contentful Paint (LCP): 5.779 seconds
[Performance][Electron Frontend Startup][10] Largest Contentful Paint (LCP): 6.147 seconds
[Performance][Electron Frontend Startup][MEAN] Largest Contentful Paint (LCP): 6.093 seconds
[Performance][Electron Frontend Startup][STDEV] Largest Contentful Paint (LCP): 0.230 seconds

Note that the existing NPM script for the browser start-up performance is renamed from performance:startup to performance:startup:browser to distinguish it from the Electron case.

A new command-line option --app (-a) is added to the extension-impact.js script that selects the example application to build for execution of the tests. Supported values are browser and electron; the default if not specified is the browser app as before.

No new package dependencies are added.

Contributed on behalf of STMicroelectronics.

How to test

See the readme in the scripts/performance/ directory for details of how to run the script. For most cases, it is sufficient just to run the NPM script from the monorepo root:

$ yarn performance:startup:electron

This will print out performance measurements with summary statistics as shown above. You may also import the traces generated by the script in the scripts/performance/profiles/electron/ directory into the Performance tab of the Chrome Developer Tools for analysis.

Also, try running the Extension Impact script with the new option to drive the Electron app:

$ cd scripts/performance
$ node ./extension-performance.js --app electron

Note that executing the extension impact script on the Electron app after having run it on the browser app will often fail to get measurements for the @theia/git extension; it throws an exception about a mismatch in node versions in the native modules:

Error: The module '/home/runner/work/theia/theia/node_modules/find-git-repositories/build/Release/findGitRepos.node'
was compiled against a different Node.js version using NODE_MODULE_VERSION 83. This version of Node.js requires
NODE_MODULE_VERSION 80. Please try re-compiling or re-installing the module (for instance, using `npm rebuild` or
`npm install`).

This can be remedied by cleaning the Git workspace and building it again before running the script.

Review checklist

Reminder for reviewers

@vince-fugnitto vince-fugnitto added electron issues related to the electron target performance issues related to performance labels Nov 22, 2021
Copy link
Contributor

@colin-grant-work colin-grant-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the most part, this works as advertized. I did run into one oddity. It's possible to get into a state where running the electron app fails because electron-rebuild erroneously believes that native modules have been build for electron. Usually at startup, there's a message to the effect that some module did not self-register. When I was in that state and tried to run this script, I just got repeated logs about inability to analyze the performance. If there's some way to detect that error and provide a more helpful message, that might be useful.

scripts/performance/common-performance.js Outdated Show resolved Hide resolved
scripts/performance/electron-performance.js Outdated Show resolved Hide resolved
scripts/performance/electron-performance.js Outdated Show resolved Hide resolved
scripts/performance/browser-performance.js Outdated Show resolved Hide resolved
scripts/performance/electron-performance.js Outdated Show resolved Hide resolved
@cdamus
Copy link
Contributor Author

cdamus commented Nov 25, 2021

Thanks, @colin-grant-work . Good points, all. I've pushed commit 60f5cfd that should address them, including adding a --debug option for logging error output from the Theia child process (I did find I often needed this to diagnose issues). This works together with detection of the common case of the native modules being built for the wrong target to exit the script early.

@cdamus cdamus mentioned this pull request Nov 25, 2021
1 task
@colin-grant-work colin-grant-work self-requested a review November 29, 2021 21:47
Copy link
Contributor

@colin-grant-work colin-grant-work left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest changes look good to me and the error output provided useful feedback where possible.

@cdamus
Copy link
Contributor Author

cdamus commented Nov 30, 2021

The force-push of commit edc22d5 just resolves the conflict in the change log.

@colin-grant-work
Copy link
Contributor

@cdamus, please rebase this branch, and then I will merge it.

- refactor a reusable module out of the startup measurement script
- create a new script to measure Electron start-up performance
- rename the measure-performance.js script to indicate that it
  measures the browser app specifically
- add an --app/-a option to the extension-impact script to drive
  the Electron app as an alternative to the Browser app. Includes
  a bit of refactoring of this script and the app package template
- incidentally fix the extension-impact.js stepping over itself
  trying to run the app while building it (was missing an await)

Contributed on behalf of STMicroelectronics.

Co-authored-by: Stefan Dirix <[email protected]>
Signed-off-by: Christian W. Damus <[email protected]>
@cdamus
Copy link
Contributor Author

cdamus commented Dec 8, 2021

Thanks, @colin-grant-work I've force-pushed 5242fbe. The only conflict was in the changelog.

@colin-grant-work colin-grant-work merged commit 7f954ac into eclipse-theia:master Dec 8, 2021
@vince-fugnitto vince-fugnitto added this to the 1.21.0 milestone Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
electron issues related to the electron target performance issues related to performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Electron start-up performance script
3 participants